1. Field of the Invention
The present invention relates to the field of card playing, and more particularly to a method and apparatus for generating improved card playing strategies.
2. Background Art
The game of poker has long been popular, both among card players and game theorists. A large number of books and papers have been written on poker playing strategies. Examples include "Winning Poker Systems" and "Computation of Optimal Poker Strategies" by Norman Zadeh (Wilshire Book Company, 1974 and Operations Research, Vol. 25, No. 4, July-August, 1977, respectively), "Poker Strategy" by Nesmith C. Ankeny (Perigee Books, 1981.), "An Optimal Strategy for Pot-Limit Poker" by William H. Cutler (American Math Monthly, Vol. 82, April 1975), and "Theory of Games and Economic Behavior" by Von Neiman and Morgtern (Princeton University Press, 1944). While the strategies described in the prior art attempt to generate strategies leading to improved returns for players, the methods used to generate the strategies are based on inexact empirical observations or serious analytic simplifications and approximations. As a result, the prior art strategies produce insatisfactory real world results.
Poker Basics
In one popular version of poker, hands consist of five cards from a 52 card deck, resulting in 2,598,960 different hands. The hands are linearly ordered in strength or "rank." There are nine general categories of hands, ranked as shown in Table 1.
TABLE 1 ______________________________________ Ranking by Categories Rank Name Example ______________________________________ 1 Straight flush J 10 9 8 7 2 Four of a kind K K.diamond-solid. K.heart. K 9 3 Full house J.diamond-solid. J.heart. J 3 3.heart. 4 Flush A 10 9 5 2 5 Straight 6.diamond-solid. 5.heart. 4 3 2.heart. 6 Three of a kind 10 10 10.diamond-solid. 9.heart. 7 7 Two pair A.heart. A 4 4.diamond-solid. 9 8 One pair 9.diamond-solid. 9.heart. K 8 6.heart. 9 No pair 3 5.diamond-solid. 6.heart.J Q ______________________________________
Within each category, hands are ranked according to the rank of individual cards, with an ace being the highest card and a 2 being the lowest card. There is no difference in rank between the four suits of cards. Table 2 shows the ranking of some example hands within the two-pair category. Because the suits of the individual cards do not matter for two pair hands (the suits become relevant only for flushes and straight flushes, since all cards in these hands must be of the same suit), no suits are shown in Table 2.
TABLE 2 ______________________________________ Relative Ranking of Some Two Pair Hands ______________________________________ Highest AAKKQ AAKKJ AAKK10 AAKK9 * * * AAQQ2 AAJJK AAJJQ * * * JJ223 101099A 101099K * * * 33226 33225 Lowest 33224 ______________________________________
All hands can be ranked in a linear ranking from highest to lowest. Because suits are all of the same value, however, there are multiple hands that have identical rankings. For example, there are four equivalent hands for each type of straight flush, four of a kind, or flush; there are over a hundred equivalent hands for each two pair variation, and there are over 1000 equivalent hands for each type of no pair hand. Accordingly, although there are over 2,000,000 possible hands, there are significantly fewer possible rankings.
Poker is characterized by rounds of card dealing and betting. Numerous variations of poker exist, including "five card draw," "five card stud," and "seven card stud." The variations generally differ in the manner in which cards are dealt and in the manner in which bets are placed.
Typically, a game starts when each player has placed an initial bet, called the "ante," into the "pot." A player must ante to be permitted to play that game. The term "pot" refers to the total accumulation of bets made during a game. Each player that has "anted" is dealt an initial set of cards. The number of cards depends on the particular variation of poker being played. In five card draw, each player is initially dealt five cards.
After the deal, the players have the opportunity to place bets. If a player places a bet, that bet must be matched ("called") or "raised" by each player that wants to remain in the game. A player who does not match a bet drops out of the game or "folds."
Each game may have several "rounds" of betting. If two or more players remain after a round of betting, either more cards are dealt, or there is a "showdown," depending on the game variation being played. A "showdown" occurs when two or more players remain in a game after the last round of betting for a game has been completed. A player wins a game of poker (also sometimes called a "hand of poker") either by having the highest ranking hand when a "showdown" occurs, or by being the last remaining player in the game after all other players have dropped out, or "folded." At a showdown, each player displays the player's hand to the other players. The player showing the hand with the highest ranking wins the pot.
FIG. 1 illustrates the sequence of events that occur in a game of five card draw poker. As shown in FIG. 1, the game begins with each player paying an ante into the pot at step 100. At step 105, each player is dealt five cards by one of the players who is referred to as the dealer. Players take turns being the dealer.
After each player has been dealt the initial set of five cards, the first round of betting occurs at step 110. In a round of betting, each player is successively given the opportunity to either "pass" (i.e. to place no bet, allowed only if no one has previously placed a bet during the round), to "bet" (i.e. to place the first bet of the betting round), to "call" (i.e. to pay an amount into the pot equal to the total amount paid by the immediately preceding bettor), to "raise" (i.e. to pay an amount into the pot greater than the amount.paid by the immediately preceding bettor), or to "fold" (i.e. to not pay anything into the pot and thereby to drop out of the game). The betting sequence typically starts with the player to the immediate left of the dealer, and then progresses in a clockwise direction.
FIG. 2 illustrates an example of a first round of betting that may occur at step 110 of FIG. 1. In the example of FIG. 2 there are three players: player A 200, player B 205, and player C 210. Player A is the dealer. In FIG. 2, the cards dealt to each player are shown under the player's name. Thus, after the deal, player A's hand is AA762, player B's hand is KK225, and player C's hand is JJ843.
Since player B is the player to the immediate left of the dealer (player A), player B begins the betting round. Player B may pass (bet nothing), or place a bet. Player B's hand contains two pairs, which player B considers to be a good first round hand. Accordingly, as shown in FIG. 2, player B bets one "bet" at step 215. In this example, betting "one bet" means that the bettor bets the maximum betting limit allowed by the rules of the particular variation of poker game being played. Two types of betting are "limit" betting and "pot limit" betting. In limit betting, the maximum betting limit is a predetermined amount. For example, a betting limit may be $2. In pot limit betting, the maximum amount that a player may bet is the total amount in the pot at the time the bet is made, including the amount, if any, that the bettor would need to put into the pot if the bettor were calling.
After player B has bet, it is player C's turn to act. Since player B has bet one bet, player C's choices are to match player B's bet ("call"), to raise, or to fold. Player C has a pair of jacks, which player C considers to be good enough to call but not good enough to raise. Accordingly, as shown in FIG. 2, player C calls at step 220 by placing an amount equal to player B's bet into the pot.
After player C has bet, it's player A's turn. Player A has a pair of aces, which player A considers to be good enough for not just calling, but raising. Player A therefore decides to raise player B's bet by one bet at step 225. Player A thus places a total of two bets into the pot--one to meet B's bet, and one to raise by one bet.
After player A raises one bet, the betting proceeds back to player B. Player B considers his two pair hand to be good enough to call player A's bet, but not good enough to reraise. Accordingly, player B calls at step 230 by putting one bet (the amount of player A's raise) into the pot so that the total amount bet by player B equals the total amount bet by player A.
After player B bets, the betting returns to player C. To stay in the game, player C must place one bet into the pot to match player A's raise. However, player C doesn't believe that player C's hand of two jacks is good enough to call player A's raise. Accordingly, player C decides to drop out of the game by folding at step 235.
After player C folds, there are no remaining un-called raises or bets. Accordingly, the first round of betting ends at step 240. Thus, after the first round of betting, there are two remaining players, player A and player B.
The size of the pot in the example of FIG. 2 after the first round of betting depends on the size of the initial ante and the betting limit of the game. Table 3 illustrates the growth in the size of the pot during the round of betting illustrated in FIG. 2 for a betting limit of $1 and for a pot limit. In both cases, it is assumed that the total ante of all three players is $1.
TABLE 3 ______________________________________ Size of Pot for Limit and Pot Limit Poker For Example of FIG. 2 Resulting Pot Resulting Pot Betting Step Action ($1 Limit) (Pot Limit) ______________________________________ 0 Ante $1 $1 1 B bets 1 bet $2 $2 2 C calls B's bet $3 $3 3 A raises by 1 bet $5 $8 4 B calls A's raise $6 $12 5 C folds $6 $12 ______________________________________
Thus, at the end of the first round of betting illustrated in FIG. 2, the resulting pot is $6 for $1 limit poker and $12 for pot limit poker.
Referring again to FIG. 1, at the end of the first round of betting at step 110, a determination is made as to whether more than one player is left in the game at step 115. If only one player is left, that player wins the pot at step 120. If more than one player is left, play continues to step 125.
At step 125, the players remaining in the game have the opportunity to discard cards from their hands and replace them with newly dealt cards. A player may discard and replace (or "draw") from 0 to 5 cards.
After the "draw" at step 125, the second round of betting takes place at step 130. The second round of betting proceeds in the same manner as the first round of betting. FIG. 3 illustrates an example of a second round of betting that occurs after the first round of betting of FIG. 2. In the example game of FIG. 3, player A and player B each drew 1 card during the draw. Player A could have drawn more cards, but player A chose to draw only one card to make it appear that player A had a better hand than player A's pair of aces. Player A discarded the lowest card of player A's hand (a 2), and was dealt a 9. Player A's resulting hand as shown in FIG. 3 is AA976.
Player B, starting off with four good cards (two pairs), also drew one card, discarding a 5 and being dealt a 7. Player B's resulting hand as shown in FIG. 3 is KK227.
The betting, in round 2, as in round 1, commences with player B. As shown in FIG. 3, even though player B has a fairly good two pair hand, player B chooses to "check" ("check" is another way to say "pass") at step 300. A check is equivalent to a pass, or to betting zero. The betting then proceeds to player A. Although player A's hand is not particularly strong, player A decides to bet 1 bet at step 305, hoping that player B will believe that player A has a strong hand and therefore fold. Making a bet with a weak hand that probably will not win in a showdown is referred to as "bluffing."
Player B does not fold, but instead raises player A by one bet at step 310. Player B thus pays two bets into the pot: one to meet player A's bet, and one to raise player A one bet. Player A, believing that player B's raise is a bluff, decides to reraise player B at step 315. Player A thus pays two more bets into the pot, one to match player B's raise and one for the reraise. Player B, not having bluffed, calls player A's reraise at step 320 by paying a bet into the pot to match player A's one bet reraise.
Player B's call of player A's reraise ends the second round of betting, leading to a showdown at step 325. The amount of money in the pot at the end of the second round of betting depends on whether the game is a limit game or a pot limit game. Table 4 shows the growth in the pot in the second round of betting for limit and pot limit games given the first round pot shown in table 3.
TABLE 4 ______________________________________ Size of Pot for Limit and Pot Limit Poker For Example of FIG. 3 Resulting Pot Resulting Pot Betting Step Action ($1 Limit) (Pot Limit) ______________________________________ 0 Beginning pot $6 $12 1 B checks $6 $12 2 A bets 1 bet $7 $24 3 B raises 1 bet $9 $72 4 A reraises 1 bet $11 $216 5 B calls $12 $324 ______________________________________
As shown in Table 4, in a pot limit game, the size of the pot increases dramatically with each pot limit bet, while the increase of the pot in a limit game is more moderate.
Referring again to FIG. 1, after the second round of betting at step 130, a determination is made as to whether more than one player is left in the game at step 135. If only one player is left, the remaining player wins the pot at step 140. If more than one player remains in the game, there is a showdown at step 145. The remaining players shown their hands, and the highest ranking hand wins the pot at step 150. In the example of FIG. 3, player B's hand of two pairs has a higher ranking than player A's hand of a pair of aces. Accordingly, player A's bluffing strategy proves unsuccessful, and player B wins the pot.
Prior Art Attempts to Generate Optimal Poker Playing Strategies
Attempts have been made in the prior art to generate optimal poker playing strategies that will provide a player with the best average economic return for any given hand dealt to the player. Many of these attempts have focused on a player's average "expected return" for taking actions such as passing, calling, betting, raising and bluffing given a particular hand of cards.
The expected return for a given action, given a particular hand of cards in a particular game circumstance, is the average return to a player for taking the action if the action were repeated many times. The expected return is the sum of the actual returns for each repetition divided by the number of repetitions. A player's overall actual return for a particular game of poker is the player's winnings (if any) from the game minus the player's investment in the game (i.e. the amount the player pays into the pot over the course of the game). For example, Table 5 shows the investment, winnings, and the net actual return for each of the players A, B, and C in the game of FIGS. 2 and 3, assuming the game is a limit game in which the limit is $1 and the initial ante is $0.33. In Table 5 bets are indicated by minus signs, and winnings by plus signs.
TABLE 5 ______________________________________ Overall Actual Returns for Players A, B, and C For Example of FIGS. 2 and 3 (in dollars) Action A B C Total Pot ______________________________________ First Round Ante -0.33 -0.33 -0.33 1 B bets 1 bet 0 -1 0 2 C calls B's bet 0 0 -1 3 A raises by 1 bet -2 0 0 5 B calls A's raise 0 -1 0 6 C folds 0 0 0 6 Second Round B checks 0 0 0 6 A bets 1 bet -1 0 0 7 B raises 1 bet 0 -2 0 9 A reraises 1 bet -2 0 0 11 B calls 0 -1 0 12 Total bet -5.33 -5.33 -1.33 Showdown 0 +12 0 Net return -5.33 +6.67 -1.33 ______________________________________
For the example game of FIGS. 2 and 3 therefore, the actual overall return for player A is -$5.33, for player B +6.67, and for player C -$1.33.
The returns shown in Table 5 are the overall returns to each player for the entire game. Expected and actual returns may also be calculated for specific parts of the game. For example, returns may be calculated for the second round of play only. In calculating returns for the second round of play, the amounts invested by the players during the first round of play may or may not be taken into account. In the case where first round investments are not taken into account, returns for the second round of betting are calculated based on the size of the pot at the beginning of the round and the amounts invested by the players during the second round. Table 6 shows the returns for the second round for remaining players A and B in the example of FIGS. 2 and 3, neglecting first round investments made by the players.
TABLE 6 ______________________________________ 2nd Round Actual Returns for Playem A and B For Example of FIGS. 2 and 3 (in dollars) Action A B Total Pot ______________________________________ Beginning Pot 6 B checks 0 0 6 A bets 1 bet -1 0 7 B raises 1 bet 0 -2 9 A reraises 1 bet -2 0 11 Bcalls 0 -1 12 Total bet -3.00 -3.00 Showdown 0 +12 Net return -3.00 +9.00 ______________________________________
The second round actual returns for players A and B for the example of FIGS. 2 and 3 are thus -$3 and +$9, respectively.
Since the payments made by players A and B into the pot are omitted when calculating the second round investments and returns in Table 5, the returns shown in Table 5 can be considered to be actual returns to players A and B for a two-player second round contest in which player A's hand is AA762 and player B's hand is CK227, and in which the beginning pot is $6. The actions that player B took in this second round of betting were to check, to raise, and to call player A's reraise. This sequence may be referred to as a "check-raise-call" sequence. Similarly, the actions that A took in the second round of betting were to bet and to reraise. This sequence may be referred to as a "bet-reraise" sequence.
More generically, from player A's point of view, the situation at the time player A first acts in round two of betting for the example of FIG. 3 is:
a) There is a certain amount in the pot, in this case, $6. PA1 b) Player A has a hand that has a specific rank. In this case, A's hand is AA762. If hands are assigned relative hand strength rankings between 0 and 1 (1 being highest), then the rank of player A's hand will be some number S between 0 and 1. (See, for example, Von Neuman and Morgenstern, "Theory of Games and Economic Behavior," Princeton University Press 1944). PA1 c) Player B has checked. Accordingly, the following sequence of actions are possible (assuming that the game is limited to one reraise):
a) Player A also checks, and there is an immediate showdown. The sequence of A's action under this option is "check." PA2 b) Player A bets, and player B calls. The sequence of A's actions under this option is "bet." PA2 c) Player A bets, and player B folds, in which case player A wins the pot. The sequence of A's actions under this option is "bet." PA2 d) Player A bets, player B raises, and player A folds. The sequence of player A's actions under this option is "bet-fold." In some cases, a "bet-fold" sequence is the result of a "bluff bet." Player A hopes to cause player B to fold with the bet, but if player B answers with a raise, player A folds. PA2 e) Player A bets, player B raises, player A reraises, and player B calls. The sequence of player A's actions under this option is "bet-reraise." PA2 f) Player A bets, player B raises, player A reraises, and player B folds. The sequence of player A's actions under this option is "bet-reraise." PA2 g) Player A bets, player B raises, and player A calls. The sequence of player A's actions under this option is "bet-call."
Although there are seven separate scenarios that may occur, there are only five possible sequences of actions for player A: i) check; ii) bet; iii) bet-fold; iv) bet-reraise; and v)bet call. Since the second sequence ("bet") is included in the third through fifth sequences, this list can further be reduced to four possible sequences of actions: i) check; ii) bet-fold; iii) bet-reraise; and iv) bet-call.
In the example of FIG. 3, the action sequence that player A chose to take was to "bet-reraise." As shown in FIG. 6, the resulting return to player A was a loss of $3 (-$3).
From FIG. 3, the returns to player A if player A had taken each of the other three action sequences can be calculated.
For the "check" sequence, the result would have been that player B would have won the pot in the showdown. A's investment in the second round would have been $0, and A's winnings would have been $0. Therefore A's net return for a "check" would have been $0.
For the "bet-fold" sequence bluff bet), A would have bet $1, B would have called, then A would have folded. A's investment would have been $1, and A's winnings would have been $0. A's net return for a bluff bet would have been -$1.
For the "bet-call" sequence, A would have bet $1, B would have raised, A would have called with a $1 bet, and B would have won the showdown. A's investment would have been $2, and A's winning $0. Thus A's net return for a "bet-call" sequence would have been -$2.
Table 7 summarizes the actual second round returns to Player A that would have resulted given the circumstances of FIG. 3 for each of Player A's four possible action sequences check, bet-fold, bet-call, and bet-reraise.
TABLE 7 ______________________________________ Second Round Returns for Player A with Different Action Sequencesfor Example of FIG. 3 Action Sequence Return ______________________________________ Check $0 Bet-fold (bluff bet) -$1 Bet-call -$2 Bet-reraise -$3 ______________________________________
From Table 7, it can be seen that by choosing the "bet-reraise" sequence in the example game of FIG. 3, player A chose the action sequence that resulted in the lowest actual return for the particular game of FIG. 3. Player A would have obtained the best possible return by following the first action sequence option: Check. If player A had known the actual outcome of the game, player A would have selected the "Check" action sequence.
However, it is impossible for player A to know, ahead of time, what cards player B holds, or what the particular outcome of a game will be. What player A knows is player A's own hand, the size of the pot, and that player B has checked. Since the specific outcome of any action sequence chosen by player A will depend on what cards B holds and how player B plays, it will be impossible for player A to predict the actual return for each action sequence in any particular game. However, if player A were able to play a large number of games in each of which player A has a hand having the same ranking S as in the example of FIG. 3, in which B checks, but in which B has a variety of hands, and if A recorded the outcome of each action sequence for each of the games, A could obtain an average expected return for each of the action sequences for the situation of a second round betting round in which A has a hand of ranking S and player B bets first and checks. Player A would then be able to determine which action sequence, in the long run, will result in the highest return for a hand of ranking S if player B uses the check-raise-call sequence.
Theoretically, by playing a large number of games for each of player A's possible hands, and by keeping track of the outcomes for each action sequence, A could calculate the expected returns for each action sequence for each possible hand for each game situation. Player A would then know the best action sequence to choose for any hand. However, given there are over 2 million possible hands, such an endeavor is unfeasible.
Prior art attempts have been made to create mathematical models of poker that could be used to obtain optimal playing strategies. However, these prior art attempts have not been directly applicable to real world poker.